Skip to content

fix!: Bump the bindings to openjd-expr 0.14, openjd-model 0.15 and openjd-sessions 0.9 - #390

Open
mwiebe wants to merge 8 commits into
OpenJobDescription:mainlinefrom
mwiebe:chore/bump-openjd-rs-crates
Open

mwiebe wants to merge 8 commits into
OpenJobDescription:mainlinefrom
mwiebe:chore/bump-openjd-rs-crates

Conversation

@mwiebe

@mwiebe mwiebe commented Nov 11, 2026

Copy link
Copy Markdown
Contributor

Fixes: no GitHub issue filed.

What was the problem/requirement? (What/Why)

The Rust bindings pin openjd-expr 0.12.0, openjd-model 0.13.0 and openjd-sessions 0.8.1. The current releases carry fixes openjd.model._v1 and openjd.expr need, including openjd-rs#486, which #383 builds on.

What was the solution? (How)

Bump to openjd-expr 0.14.0, openjd-model 0.15.0 and openjd-sessions 0.9.0. Cargo.lock also moves serde-saphyr to 2.1.1 and granit-parser to 2.1.0. THIRD-PARTY-LICENSES.txt is regenerated (version lines only).

Compile fixes:

  • INT and CHUNK[INT] range list elements are IntOrFormatString (#486). The template range getter gives each as int or FormatString, as FLOAT already did and as the pure-Python model does.
  • Counts are u64 (#482). __len__ converts, raising OverflowError where it can't fit.
  • Path mapping goes through PathMapper (#472). HostContext.with_rules and PathMappingRule.apply raise ValueError for an invalid rule. Session.extend_path_mapping_rules raises SessionError rather than dropping the error.
  • The pure-Python model builds its EXPR profile inside its try, so an invalid rule raises ExpressionError.

What is the impact of this change?

Behavior changes, each pinned by a test and described in specs/:

  • Path mapping (#472, #483): sources must be absolute; the rule with the most names wins, WINDOWS over POSIX at equal depth, last wins for one location; a UNC share is one component; WINDOWS compares non-ASCII letters case-insensitively; . and empty components are ignored; no doubled separator after the destination; paths and rule paths over 32,768 bytes are refused.
  • RangeExpr is kept in canonical form (#469): str, repr and ranges() give canonical runs, == compares values, any 64-bit value but the most negative is allowed, and parsing accepts 50,000 elements; more than 50,000 canonical runs raise (#470). NONCONTIGUOUS chunks render in canonical form, so [1, 2, 4, 6] is 1,2-6:2.
  • Unresolved list comprehensions are typed list[T | nulltype], and [[range_expr]] is allowed (#462).
  • An empty PATH join behaves like pathlib (#516); escape_format_string("{}}") round-trips (#515).
  • A FLOAT range element's format string is checked at decode; an INT element that doesn't resolve to an integer fails create_job (#486).
  • YAML: tRUE, fALSE and nULL are strings, and core tags such as !!float 1.5 decode (serde-saphyr 2.1).
  • The session working directory is 8 random characters (#444); its rules file is written in precedence order.

Differences from the pure-Python reference that this leaves:

  • IntRangeExpr doesn't write canonical form (a strict xfail in model_v1/test_known_gaps.py).
  • openjd-sessions' path mapping accepts relative sources, picks the first rule on a tie and folds only ASCII letters.

No strict xfail tracking an openjd-rs gap XPASSes, so none are removed. #383's two strict xfails XPASS once it is rebased on this.

How was this change tested?

See DEVELOPMENT.md for information on running tests.

  • Have you run the unit tests? Yes, on Linux with Python 3.12 and 3.10: 6385 passed, 26 skipped, 9 xfailed, 94.3% coverage. ruff, black --check and mypy pass. cargo fmt --check, build --all-targets, clippy -D warnings, test, test --doc, doc -D warnings, cargo deny and scripts/check_third_party_licenses.sh pass.
  • openjd-cli (318) and openjd-sessions-for-python (1005) tests pass, as on mainline. With openjd-cli from source, conformance failures go from 32 to 25, none new.

Was this change documented?

  • Are relevant docstrings in the code base updated? Yes, and the three interface specs.

Is this a breaking change?

Yes, as listed above. Callers must give absolute path mapping sources, and can't rely on rule order, RangeExpr text or the working directory name.

Does this change impact security?

No. Path mapping now caps path length and rule-set size.


By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

…enjd-sessions 0.9

Move rust-bindings to openjd-expr 0.14.0, openjd-model 0.15.0 and
openjd-sessions 0.9.0. Cargo.lock changes only those three crates and
what they pull in: serde-saphyr 1.2.0 -> 2.1.1 and granit-parser
1.2.0 -> 2.1.0.

The new crates needed these binding changes:

- INT and CHUNK[INT] range list elements are now IntOrFormatString
  (openjd-rs#486). The template-side range getter exposes each element
  as an int or a FormatString, as FLOAT elements already were and as
  the pure-Python reference does.
- Counts are u64 on every target (openjd-rs#482). The binding stores
  and passes them as u64, and __len__ converts to usize, raising
  OverflowError where it doesn't fit.
- Path mapping rules apply through a PathMapper (openjd-rs#472).
  HostContext.with_rules builds one and raises ValueError for an
  invalid rule, PathMappingRule.apply builds a one-rule mapper, and
  Session.extend_path_mapping_rules raises SessionError instead of
  dropping the new Result.

Tests changed for behavior the crates changed on purpose:

- A list comprehension over an unresolved value has element type
  T | nulltype, since a run may keep nothing (openjd-rs#462).
- A NONCONTIGUOUS chunk renders in the canonical form of Template
  Schemas 3.4.1.1.1, so [1, 2, 4, 6] is "1,2-6:2" (openjd-rs#469). The
  pure-Python reference still writes "1,2,4,6"; a strict xfail in
  model_v1/test_known_gaps.py records that.
- A FLOAT range list element's format string is checked at decode
  time (openjd-rs#486), as the pure-Python reference already did.

New tests pin the other visible changes: INT and CHUNK[INT] format
string elements, relative path mapping sources rejected and the rule
with the most names winning in any order (openjd-rs#472), empty PATH
join operands (openjd-rs#516), and mixed-case tRUE and nULL read as
strings (openjd-rs#487). No strict xfail tracking an openjd-rs gap
XPASSes, so none are removed.

The specs and the type stub's docstrings describe the new behavior.
No public signature changed.

BREAKING CHANGE: PathMappingRule.apply and HostContext.with_rules raise
ValueError, and Session raises SessionError, for a rule whose POSIX or
WINDOWS source_path is not absolute. When several rules match, the one
whose source path has the most names applies, not the first in the
list. IntTaskParameterDefinition.range and
ChunkIntTaskParameterDefinition.range list elements can be
FormatString as well as int. A NONCONTIGUOUS chunk value is written in
canonical form. An unresolved list comprehension's type is
list[T | nulltype].

Signed-off-by: Mark Wiebe <399551+mwiebe@users.noreply.github.com>
Regenerated with scripts/check_third_party_licenses.sh --update. Only
the version lines of openjd-expr, openjd-model, openjd-sessions,
serde-saphyr and granit-parser changed.

Signed-off-by: Mark Wiebe <399551+mwiebe@users.noreply.github.com>
Review of the bump found visible changes that had no test or note:

- openjd.expr.RangeExpr is kept in canonical form since openjd-expr
  0.13.0 (openjd-rs#469): str, repr and ranges() give the canonical
  runs, and equality is by value. Every constructor caps the canonical
  form at 50,000 runs (openjd-rs#470). New tests pin these, and the
  strict xfail for the pure-Python IntRangeExpr also covers from_str.
- Rules for the same location resolve last-wins (openjd-rs#472). A new
  test pins it.
- An INT range list format string that doesn't resolve to an integer
  fails create_job (openjd-rs#486). A new test pins the message.
- The pure-Python model's EXPR evaluation builds its profile with
  HostContext.with_rules, which now raises ValueError for a relative
  source path. Build the profile inside the existing try, so the
  failure reaches callers as the model's ExpressionError like any
  other evaluation error, and pin it.

BREAKING CHANGE: str(), repr(), ranges() and == of openjd.expr.RangeExpr
follow the canonical form, and a RangeExpr with more than 50,000
canonical runs raises RangeExprError. Between path mapping rules for
the same location, the last one given applies. In the pure-Python
model, an EXPR expression evaluated under a path mapping rule whose
source path is not absolute raises ExpressionError. YAML scalars
spelled tRUE, fALSE or nULL are strings. An unresolved list
comprehension whose filter can never be a boolean gives
list[nulltype] rather than raising.

Signed-off-by: Mark Wiebe <399551+mwiebe@users.noreply.github.com>
…e bump

A second review of the bump found more visible changes with no test or
note:

- No separator is added after a path mapping destination that already
  ends with one (openjd-rs#483), so /dst/ maps /mnt/a/b to /dst/a/b.
- A path longer than 32,768 bytes matches no rule, and a rule with a
  longer source or destination path is invalid (openjd-rs#472).
- escape_format_string escapes a lone "{" before "}}", so "{}}" now
  round-trips (openjd-rs#515).

New tests pin each one, and the expr and sessions specs describe them,
including that a rule added by extend_path_mapping_rules replaces an
existing rule for its location.

The YAML scalar test said the change came from openjd-model 0.15.0. It
comes from serde-saphyr 2.1, which Cargo.lock resolves now that
openjd-model requires serde-saphyr 2, and which openjd-rs adopted in
#487. Reword it. Use "names", as the specs do, in the
extend_path_mapping_rules docstring, and name IntRangeExpr rather than
only from_list in the known-gap reason.

BREAKING CHANGE: a path longer than 32,768 bytes matches no path
mapping rule, and a rule whose source or destination path is longer
raises ValueError (SessionError in a Session).

Signed-off-by: Mark Wiebe <399551+mwiebe@users.noreply.github.com>
…he bump

A third review of the bump found more visible changes:

- The 32,768-byte path limit (openjd-rs#472) has three outcomes, not
  one. PathMappingRule.apply returns no match for a longer path, the
  apply_path_mapping expression function raises ExpressionError, and a
  session errors on a longer PATH or LIST[PATH] value. The specs said
  only "matches no rule". Correct them, and pin apply_path_mapping.
- Between a POSIX and a WINDOWS rule with the same number of names, the
  WINDOWS rule applies in any order (openjd-rs#472). Pin and document it.
- The session working directory is named with 8 random characters and
  no part of the session ID (openjd-rs#444), as the pure-Python
  reference names it. Pin and document it.
- A YAML 1.2 Core tag on a scalar, such as "!!float 1.5", decodes with
  serde-saphyr 2.1. With 1.2 it failed. Pin it.

Not pinned here because this repo's tests can't reach them, or they
only remove a crash: openjd-sessions 0.9.0 maps every LIST[PATH] value
when the session has a base symbol table (openjd-rs#483); a malformed
openjd_unset_env line no longer also unsets a variable named after the
parse error, and redacting a value that starts with a multibyte
character no longer panics (openjd-rs#515); with_number no longer
overflows on i64::MIN (openjd-rs#515); and CHUNK[INT] chunking no
longer overflows or lists every value for huge ranges (openjd-rs#473).

BREAKING CHANGE: apply_path_mapping raises ExpressionError for a path
longer than 32,768 bytes, and a session errors on a PATH or LIST[PATH]
value that long. A WINDOWS path mapping rule applies over a POSIX rule
with the same number of names. Session.working_directory no longer
contains the session ID.

Signed-off-by: Mark Wiebe <399551+mwiebe@users.noreply.github.com>
A fourth review of the bump found that the PathMapper rewrite
(openjd-rs#472) also changed how rules match components, as How Jobs
Are Run, "Path Mapping" item 2 requires:

- A UNC share is one component and must match as a whole, so a rooted
  path such as \srv\share\x no longer matches a \\srv\share rule.
- WINDOWS components compare the way Windows compares file names,
  non-ASCII letters included, so C:\Ünï matches C:\üNÏ. The
  pure-Python reference in openjd-sessions folds only ASCII letters.
- "." and empty components are ignored when matching and left out of
  the mapped path, so /mnt/./a/b matches a /mnt/a rule, and
  /mnt/a/./b//c maps to /d/b/c.

Pin each with a test and describe them in the expr spec, with a pointer
from the sessions spec.

BREAKING CHANGE: a rooted WINDOWS path no longer matches a UNC share
rule, WINDOWS rules match non-ASCII letters case-insensitively, and "."
and empty components are ignored when matching and dropped from the
mapped path.

Signed-off-by: Mark Wiebe <399551+mwiebe@users.noreply.github.com>
Reading the upstream commits behind the bump turned up three more
visible openjd-expr changes:

- RangeExpr values can be any 64-bit integer except the most negative
  one (openjd-rs#469). Before, magnitudes of 2**62 and up raised an
  overflow error.
- Parsing accepts up to 50,000 comma-separated elements (openjd-rs#470).
  Before, the cap was 10,000.
- A range_expr inside two list levels, [[range_expr('1-3')]], is two
  levels deep (openjd-rs#462). Before, it raised "Lists may be nested at
  most 2 levels deep".

Each is a relaxation, so nothing that worked before now fails. Pin them
and describe the first two in the expr spec.

Signed-off-by: Mark Wiebe <399551+mwiebe@users.noreply.github.com>
The expr spec, and the body of the commit that pinned component
matching, cited How Jobs Are Run, "Path Mapping" item 2. The rule is
item 2 of its "Applying Path Mapping Rules within a Job Template"
subsection. The working directory test said the new name came with
openjd-sessions 0.9.0; openjd-rs#444 first shipped in 0.8.2, which
this repo skipped. Move that test to its own file, since it is not
about path mapping rules.

Signed-off-by: Mark Wiebe <399551+mwiebe@users.noreply.github.com>
@github-actions

Copy link
Copy Markdown

Claude review · advisory

Reviewed b997af4 (full review): 0 findings.

Open: 0 blocking · 0 should-fix · 0 nit. ✅ Nothing blocking.

Fix or reply to each thread; the next revision's review re-checks open threads and resolves those it agrees are handled. Resolving a thread yourself also closes it. Later revisions review only what changed.

@mwiebe
mwiebe enabled auto-merge (rebase) November 12, 2026 00:24

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants